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Abstract 

A stationary Gaussian process is said to be long-range dependent (resp. anti-persistent) 
if its spectral density /(A) can be written as /(A) — |A|~^'*gi(|A|), where < d < 1/2 
(resp. —1/2 < d < 0), and g is continuous. We propose a novel Bayesian nonparametric 
approach for the estimation of the spectral density of such processes. Within this approach, 
we prove posterior consistency for both d and g, under appropriate conditions on the prior 
distribution. We establish the rate of convergence for a general class of priors, and apply 
our results to the family of fractionally exponential priors. Our approach is based on the 
true likelihood function, and does not resort to Whittle's approximation, which is not valid 
in a long memory set-up. 

Key-words: Bayesian nonparametric; consistency; FEXP priors; Gaussian long memory 
processes; rates of convergence 

1 Introduction 

Let X = {Xf, t = 1, 2, . . . } be a real-valued stationary zero-mean Gaussian random process, 
with spectral density /, and covariance function 7/(t) = E{XtXt+T), so that 



7/(t)= / fWe'^^dX (t = 0,±1,±2,...). 



(1.1) 



This process is long-range dependent (resp. anti-persistent) if there exist C > and a value 
d, < d < 1/2 (resp. -1/2 < d < 0), such that f{X)\X\^'^ -> C when A ^ 0. This may be 
conveniently rewritten as /(A) — A^^'^gdAj), where g : [0, n] -^ R+ is a continuous function. 

Interest in long-r ange dependen t and anti-persistent time series has i ncreased steadily in the 
last fifteen years; see iBeranI ( 1994 ) for a comprehensive introduction and iDoukhan et al.l ( 2003 ) 
for a review of theoretical aspects and fields of applications, including telecommunications, eco- 
nomics, finance, astrophysics, medicine and hydrology. Resea rch in parametric inference f or lon g 

and intermediate memory pr ocesses have been pioneered by Mandelbr ot and Van Ne ss (196q)_j 

Mandelbrot and Wallisl(ll969(). and contin ued bvlFox and Taaaul(,1986. ). lDahlhausl(|l989. ). .Giraitis and Taaau 
( 19991 ) . iGeweke and Porter-Hudakl(jl983l) . and lBeranI (J1993I ). among others. Unfortunately, para- 
metric inference can be highly biased under mis-specificat ion of the true m odel. This limitation 
makes semiparametric approaches particularly appealing (|Robinsonl . 119951 ). 
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For instance, under the representation /(A) = |A|~^''g(lA|), one would like to estimate d 
as a measure of long-rang e dependenc e , with out resorting to parametric assumptions on the 
nuisance par a meter g; see iLiseo et al.l ( 2001r) for a Bayesian approach to this problem, and 



Bardet et alJ (|2003f) for an exhaustive review of classical approaches. However, practically all 
the existing procedures either exp loit the regressi on structure of the log-spectral density in a 
small neighborhood of the origin ( Robinsonl. 1995 ). or use an approximate likelihood function 



based on Whittle's approximation ( Whittlel . ll962l) . where the original vector of observations X„ 



{Xi , ^2, . . . , Xn) gets transformed into the periodogram J(A) computed at the Fourier frequencies 
Xj — 27rj/n, j = l,2,...,n, and the artificial observations /(Ai), . . . ,/(A„) are, under short 
range dependence, approximately independent. Unfortunately, Whittle's approximation is not 
valid in in the presence of long range dependence, at least for the smallest Fourier frequencies. 

We propose a Bayesian nonparametric approach to the estimation of the spectral density 
of the stationary Gaussian process based on the true likelihood, without resorting to Whittle's 
approximation. We study the asymptotic properties of our procedure, including consistency 
and rates of convergence. Our study i s base d on standard tools for an asymptotic analysis of 
Bayesian approaches, e.g. iGhosal et al.l ([20001), i.e. quantities of interest are the prior probability 



of a small neighborhood around the true spectral density, and some kind of entropy measure for 
the prior distribution. Most technical details differ however, as the observed process is long-range 
dependent. 

The paper is organised as follows. In Section [2l we introduce the model and the notations. 
In Section [21 we provide a general theorem that states sufficient conditions to ensure consistency 
of the posterior distribution, and of several Bayes estimators. We also intr oduce the c lass o f 



FEXP (Fractional Exponential) priors, based on the FEXP representation of [Robinsonl (|l99ir) . 
and show that such prior distributions fulfil these sufficient conditions for posterior consistency. 
In Section [4j we study the rate of convergence of the posterior in the general case, and specialise 
our results for the FEXP class. Section \5\ gives the proofs of the main theorems of the two 
previous Sections. Section [SI discusses further research. The Appendix contains several technical 
lemmas. 

2 Model and notations 

The model consists of an observed vector X„ — {Xi, . . . , Xn) of n consecutive realizations from 
a zero-mean Gaussian stationary process, with spectral density /, which is either long-range 
dependent, short-range dependent, or anti-persistent. The likelihood function is 

<^(X„;/) = (27r)-"/2|T„(/)rV2cxp{-ix*,T„(/)-iX„} (2.1) 

where T'„(/) is the Toeplitz matrix associated to 7/, see (|l.ll) . i.e. T„(/) = [jf{j — fc)]i<j,/c<n- 

This model is parametrised by the pair {d,g), which defines / = F[d,g) through the factori- 
sation 

F: (-l/2,l/2)xC"[0,7r] ^ F 

{d,g) ^ /:/(A) = |A|-2V|A|), 

where C^PjTt] is the set of continuous functions over [0,7r], and F denotes the set of spectral 
densities, that is, the set of even functions / : [— 7r,7r] -^ R+ such that /_ |/(A)| dX < -l-oo. 

The model is completed with a nonparametric prior distribution tt for {d,g) G (—1/2, 1/2) x 
C*'[0,7r]. (There should be no confusion whether tt refers to either the constant or the prior 
distribution in the rest of the paper.) All our results will assume that the model is valid for 



some true' parameter {do, go), associated to some 'true' spectral density /o = F{do,go), where 
do G (—1/2, 1/2); conditions on go are detailed in the next section. 

We introduce several pseudo-distances on J". The KuUback-Leibler divergence for finite n is 
defined as 



KLMo;f) 



1 



</5(X„; /o) {log(^(X„; /o) - log(/?(X„; /)}(iX„ 



= ^ {tr [rn(/o)T;-'(/) - In] - logdct(T„(/o)r-i(/))} 



where I„ represents the identity matrix of order n. Letting n -^ oo, we can define, when it exists, 
the quantity 



KL^{fo;f) 



1 

An 



/o(A) 
/(A) 



l-lo( 



/o(A) 
/(A) 



dA. 



We also define a symmetrised version of KLn, i.e. 

K{fo, f) = KL„ifo; f) + KLM; /o); 

and its limit as n — > cxd: 



HfoJ)^ 



1 

47r 



/o(A) ./(A) 
/(A) ^ /o(A) 



-2 



dA = 



1 f7/o(A) iV/(A)^_. 
2vrio I /(A) V /o(A)''''- 



For technical reasons, we define also the pseudo-distance 



1 



and its limit as n 



&n(/o,/) = -tr {TMr'TMo-DY 



KfoJ)--r (^-l\ dX. 



An J_, V /(A) 



Final ly, we consider the L^ distance between the spectral log-densities (JMoulines and Soulier . 



eifoJ) 



{log foiX)- log f{X)ydX. 



(2.2) 



For the models considered in this paper, this distance always exists, whereas the L^ distance 
may not. 



3 Consistency 

We first state and prove the strong consistency of the posterior distribution under very general 
conditions on both n and /o = F(do,go), i-C. as n — > oo, and for e > small enough, 

P^[A|X„] ^ 1, a.s., 

where P'^[.|X„] denotes posterior probabilities associated with the prior tt, and 

Ae = {(d,.9) e (-1/2,1/2) X C°[0,^] : h{fo,F{d,g)) < e}. 

From this, we shall deduce the consistency of Bayes estimators of / and d. Finally, we shall 
introduce the class of FEXP priors, and show that they allow for posterior consistency. 



3.1 Main result 

Consider the following sets: 

g{m, M) = {5 e C°[0, tt] : TO < g < M) 
g{m,M,L,p) = {g(ig{m,M):\g{\)-g{\')\<L\\ ->!]"} 
g{t,m,M,L,p) = [-l/2 + i,l/2-t] xe(m,A/,L,p) 

for p e (0, 1], L > 0, rri < Af , t G (0, 1/2). Restricting the parameter space to such sets makes 
the model identifiable (boundedness of g, provided to > 0), and ensures that normalized traces 
of products of Toeplitz matrices that appear in the distances defined in the previous section 
converge (Holder inequality). 

We now state our main consistency result. 

Theorem 3.1. For e > small enough 

P'^iAIXJ^l, a.s. 

provided the following conditions are fulfilled: 

1. There exist t,m,M,L > Q, p £ (0,1], such that the set g{t,m,M,L, p) contains both the 
pair {do, go) that defines the true spectral density /o = F{do,go) and the support of the 
prior distribution tt. 

2. For all e > 0, t^{Bs) > 0, where B^ is defined by 

Be = {(rf, 9) e Q[t, TO, Af, i, p) : h{fo, F{d, g)) < e, 16\do ~d\<p+l-t}. 

3. For £ > small enough, there exists a sequence Tn such that -niTn) > 1 — e^'"", r > 0, 
and a net (i.e. a finite collection) 

Hn C {{d, g) G [-1/2 + f , 1/2 - t] X g{m, M, L, p) : /i(/o; F(d, g)) > e/2} 

such that, forn large enough, for all {d,g) G J-'nf^A'^, f — F{d,g), there exists {di,gi) G T-Ln, 
fi = F{di, gi), such that 8{di — d) < p + 1 — t, f < fi, and: 



-dX<h{fo,fi)/A- 



(a) if8\d,-do\<p+l-t, 

}_ r iu-DW 

2^ 7_, /o(A) 

(b) tf8{d,-do)>p+l-t, 

b{f^J)<b{foJ^)\\oge\~'; 

(c) otherwise, if 8 {do — di) > p + I — t, 

1 r (^^-•^)WdA<6(/„/o)|log£| 



27ry_, /.(A) 
4. The cardinality Cn of the net T-Ln defined above is such that logC„ < ne/log{e). 

A proof is given in Section [STTI Note that, in the above definition of the net Hn, the | loge[ 
terms are here only to avoid writing inequalities in terms of awkward constants in the form m/M . 
If need be, we can replace the | logej by the correct constants as expressed in Appendix B. The 
definition of the above entropy is non-standard. The interest in expressing it in this general but 
non-standard form lies in the difficulty in dealing with spectral densities which diverge at 0. In 
practise, the way one constructs the net Tin should vary according to the form of the prior on 
the short memory part g. 



3.2 Consistency of point estimates 

As explained in 521 we focus on the quadratic loss function £ with respect to the logarithm of 
the spectral density. The corresponding Bayes estimator is 

d = E^[d\K,,], g :A^exp{£;" [log 5(A) |X„]}, f^F{d,g). 

Often, the real parameter of interest is d, and 5 is a nuisance parameter. Consistency for d can 
be deduced from Theorem |3JJ 



D 



Corollary 1. Under the assumptions of Theorem \3.1[ for e > small enough, 

P^ [{\d - do\ > e}\X,,] ^ 

and d — > do as n -^ 00. 

Proof. Lemma [TUl see Appendix iDl implies that 

P"[^^|X„] > P" [{\d ~do\> e'} |X„] ^ a.s. 
as n -^ +CXD, for some e' > and, by Jensen's inequality, 

(d-do)^ <^"[(d-rfo)^|X„] ->0, a.s. 

Consistency results for a point estimate of / can also be deduced: 
Corollary 2. Under the assumptions of Theorem \3.1[ one has, as n -^ 00, 

^(/o,/)^0, a.s. 

Proof For / = F{d,g), /o = F{do,go), one has i{fo, f) < h{fo, /), since x^ < e"' + e"^ - 2 for 
all X, and ^(/o,/) < C for some well chosen constant C, provided g, go & Q{m,M). Thus, by 
Jensen inequality, and for all e > 0, 

^(/o,/) <i?"[^(/o,/)|X„] <£ + CP-[A^|X„]. 

D 

3.3 The FEXP prior 

Following iHurvich et al.l ( 2002 ). we consider the FEXP parameterisation of spectral densities. 



i.e. / — F{d, k, 9), where 

F:T -^ T 
{d,k,9) ^ /:/(A) = |l-e'^r2'iexpJ^0,cos(jA)l. (3.1) 

and r = (-1/2 + t, 1/2 - t) x {u^J^jfc} x M'^+i}, for some fixed t e (0, 1/2). This FEXP 
representation is equivalent to our previous representation / = F{d,g), provided g = ■0~''e"', 

w{X) = {Ei=o^jCos(iA)} and i/;(A) = |1 - e^^p/A^ = 2(1 - cosA)/A2 for A ^ 0, V^(0) = 1. 



The function ip is bounded, infinitely differentiable and positive for A G [0, tt]. Thus g and w 
share the same regularity properties, i.e. w is bounded and Holder with exponent p implies that 
g is bounded and Holder with exponent p, and vice versa. Under this parameterisation, the 
prior distribution it is expressed as a trans-dimensional prior distribution on the random vector 
{d,k,9), which, for convenience, factorises as TTd{d)Trk{k)TTg{9\k). 
We assume that tt puts mass one on the following Sobolev set: 

5(/3, L)=l{d,k,9)eT:Y^ 6]{j + \f^ < L I (3.2) 

for some /3 > 1/2, L > 0. This ensures that the Fourier sum w, and thus the short-memory 
component g of the spectral density /, as explained above, belong to some set Q{m^ M^ L' , p), 
i.e., both w and g are bounded and Holder, for p < /3 — 1/2. To see this, note that, for 

(d,fc,0)G5(/3,i): 

J-0 j-0 j^O 

+c>o 

< L + ^0- + l)2'^-2'3<+oo, (3.3) 

3=0 

provided 2r — 2/3 < —1. By taking r = 0, one sees that w is bounded, and by taking r = p, for 
any p, < p < /3 — 1/2, one sees that w is Holder, with coefficient p, since, for A, A' G [— tt, tt], 

k 

\w{X)-w{X')\ < 2^|0j-| X |{cos(Aj) -cos(A'j)}/2r 

3=0 

Finally, we assume that tt assigns positive prior probability to the intersection of 5(/3, L) with 
any rectangle set of the form 

k 

{ad,bd) X {k} X ]^(ae^,^). 
i=i 
Alternatively, one could assume that the support of n is included in a set of the form 
{{d,k,9) G T : X]i=o l^jlJ'' — ^}- However Sobolev sets are more natural when dealing with 
rates of convergence, see Section 14.21 and are often considered in the non parametric literature, 
so we restrict our attention to these sets. 

In the same spirit, we assume that the true spectral density admits a FEXP representation 
associated to an infinite Fourier series. 



/o(A) - |1 - e^^r^'i" exp <^ ^ 9o, cos(jA) \ , 



j=o 



i.e., /o = F{dQ, go) with go = ip ''"e""' and u'o(A) = I J2j:^ ^oj cos(jA) >. In addition, we assume 
that Wo satisfies the same type of Sobolev inequality, namely 

^o^J2 <■ (j + 1)'^ < i < +00, (3.4) 

3=0 



which, as explained above, imphes that go G G{m, M, L, p), for some well chosen constants 
m,M,L, p. Note that it is essential to have a strict inequality in (|3.4p . i.e. Lq < L. 

Theorem 3.2. Letir be a prior distributiornTd{d)T:k{k)TTeiO\k) which fulfils the above conditions, 
and, in addition, such that 7rfc(fc) < exp(— Cfclogfc) for some C > and k large enough. Then 
the conditions of Theorem \3.1\ are fulfilled, and the posterior distribution is consistent. 

Proof. Condition 1 of Theorem [3TT] is a simple consequence of (j3.4p and (13.21) . as explained above. 
For Condition 2, we noted, see ([33]), that ^+!^ 9^j{j + l)^'^ < L implies that J2t=o I^Ojl < L' < 

+CX). Let k such that J2T=k+i l^ojl < e/14, = (6*0, ...,6'fe) such that Y^^q \0oj - Oj\ < e/14, 
d such that \d — do\ < e/7, and let / — F{d, k,6). Using Lemma [HI see Appendix iDl one has 
h{f, /o) < £. Note that it is sufficient to prove that Tr(Be) > for e small enough, hence we 
assume that e/7 < (p + 1 — t)/16. Thus, Condition 2 is verified as soon as the intersection of 
S{f3, L) and the rectangle set 

k 

[do - e/7, do + e/7] x {k} x [|[6ioj - s/Uk, Oqj - e/Uk] 

3 = 1 

is assigned positive prior probability. Now consider Condition 3. Let e > and take 

Tn = {{d,k,e)eS{P,L): fc<fc„}, 

where fc„ = [an/lognj, for some a > 0, so that, for some r depending on a, i^{J-n) < TTk{k > 
k„) < e""''. Let / = F{d, k, 9), f, = {2ef^F{d„ k, 9i), such that k < k^, d^ - ce < d < d,, and 
J2j=o \^j ~ %l — "-^j ^'^^ some c > 0, then 

■^^^^ = (2e)-=" [2(1 - cosA)]'*'"'^exp { V(0j - %)cosO-A) } < 1, 



/.(A) 




and 



/(A) ^ ^_| ^^ \\cer,-ce „-2ce 



/.(A) 



> (l-cosA)'=''2-"-'e- 



If c is small enough, fi — f verifies the three inequalities considered in Condition 3. The number 
Cn of functions fi necessary to ensure that, for any / in the support of tt, at least one of them 
verify the above inequalities, can be bounded by, for n large enough, and some well chosen 
constant C, 

< exp {3an [1 + (log a — log log n) / log n] } 

< exp {6an} 

so Condition 4 is satisfied, provided one takes a — £/61oge. 

D 

A convenient default choice for tt is as follows: Tr^ is uniform over (—1/2 + i, 1/2 — t), Hk 
is Poisson, and 7rg|fe has the following structure: the sum S — J2j=o^^i^ + 1)^^ ^^® ^ Gamma 
distribution truncated to interval [0,L], independently of S, the vector (0g,0^2^'^, . . . , 0^(fc + 
1)'^^)/S is Dirichlet with some coefficients ai^k, ■ ■ ■ , ctk.k, and the signs oi do, ■ . ■ ,0k have equal 



probabilities. In particular one may take aj^k — 1 for all j < k, or, if one needs to generate 
more regular spectral densities, aj^k = j~'^ , for some fixed or random k > 0. Another interesting 
choice for the prior on 9 is the following truncated Gaussian process: for each fc, and each j < k, 
9j ^ A/'(0,Tq(1 + j)~^^) independently apart from the constraint, for some fixed, large L > 0: 

Note that we can easily restrict ourselves to the important case d > 0, i.e. processes having 
long or short memory but not intermediate memory. 

4 Rates of convergence 

In this section we first provide a general theorem relating rates of convergence of the posterior 
distribution to conditions on t he prior. These con ditions are, in essence, similar to the conditions 
obtained in the i.i.d. case (e.g. lGhosal et al.l . l2000l) : i.e. a condition on the prior mass of Kullback- 



Leibler neighborhoods of the true spectral density, and an entropy condition on the support of 
the prior. We then present results specialised to the FEXP prior case. 

4.1 Main result 

Theorem 4.1. Let (m„) be a sequence of positive numbers such that Un — > 0, nu„ — > +cx}, and 
Bn a sequence of balls belonging to Q(t, m, M, L, p), and defined as 

Bn = {(d,g) : KLMo;F{d,g)) < u,Ji, 6„(/o,F(d, g)) < Un, do < d < do + 5} , 

for some 6, L > 0, < m < M , p G (0, 1] . Let ir be a prior which satisfies all the conditions of 
Theorem \3.1l and, in addition, such that: 

L For n large enough, 7r(S„) > exp(— nu„/2). 

2. There exists e > and a sequence of sets Tn C {{d,g) : h{F{d,g), /o) < e}, such that, for 
n large enough, 

TT (J-^ n {{d,g) : h{F{d,g)Jo) < e}) < exp(-2nu„). 

3. There exists a positive sequence (Sn), £^ > u„, £^ -^ 0, ne^ > Clogn, for some C > 0, 
satisfying the following conditions. Let 

Vnj = {{d,g) e Tn; ell < hMo,F{d,g)) < el{l + l)}, 

with Iq < I < In, with fixed /q > 2 and In — [e^/e^] — 1. For each I — Zq, • • • ,ln, there 
exists a net (i.e. a finite collection) T-Ln.i C Vn.i, with cardinality Cn,i, such that for all 
f = F{d,g), {d,g) G Vn,h there exists fi^i = F{di^i,gij) e H„,; such that f,j > f and 

< g^.lix) - g{x) < lslg.u/32 < d,,i -d< lel{\ogny\ 

where 

logC„,/ < ne^l", with a < 1. 



'C'net 



,/16 



(4.1) 



Then, there exist C, C" > such that, for n large enough, 

ES [P'' {hMo,F{d,g)) > loel\X^)] < Cn~^ + 2e 

A proof is given in Section 15.21 

The conditions given in Theorem 14.11 are similar in spirit to those considered for rates of 
convergence of the posterior distribution in the i.i.d. case. The first condition is a condition on 
the prior mass of KuUback-Leibler neighborhoods of the true spectral density, the second one is 
necessary to allow for sets with infinite entropy (some kind of non compactness) and the third 
one is an entropy condition. The inequality (14.11) obtained in Theorem 14.11 is non asymptotic, 
in the sense that it is valid for all n. However, the distances considered in Theorem 14.11 heavilv 
depend on n and, although they express the impact of the differences between / and /o on the 
observations, they are not of great practical use. For these reasons, the entropy condition is 
awkward and cannot be directly transformed into some more common entropy conditions. To 
state a result involving distances between spectral densities that might be more useful, we need 
to consider some specific class of priors. In the next section, we obtain rates of convergence in 
terms of the £ distance for the class of FEXP priors introduced in Section 15751 The rates obtained 
are the optimal rates up to a (log n) term, at least on certain classes of spectral densities. It is 
to be noted that the calculations used when working on these classes of priors are actually more 
involved than those used to prove Theorem 14.11 This is quite usual when dealing with rates of 
convergence of posterior distributions, however this is emphasized here by the fact that distances 
involved in Theorem 14. II are strongly dependent on n. The method used in the case of the FEXP 
prior can be extended to other types of priors. 



4.2 Rates of convergence for the FEXP prior 

We apply Theorem 14. II to the class of FEXP priors introduced in Section [531 Recall that under 
such a prior a spectral density / is parametrised as / = F(d, k, 9), see p.ip . We make the same 
assumptions as in Section 13.31 In particular, the prior 7r(d, k,9) factorises as ■Kd{d)'nk{k)'i^e{9\k), 
the right tail of tt/c is such that 

exp{— Cfclogfc} < TTkik) < exp{— C'fclogfc}, 

for some C, C > 0, and for k large enough, and there exists (3 > 1/2 such that the Sobolev set 
S{(3, L) contains the support of vr. The last condition means that S = X],-o ^j (j + l)^*^ ^ P' F] 
with prior probability one. In addition, we assume that the support of tt^ is [—1/2 + t, 1/2 — t], 
and, for d G [—1/2 + i, 1/2 — t], TTd{d) > Q > 0. Similarly, we assume that TiQ\k is such that 
the random variable S — 'Y1i'j=q ^?(j + 1)^ is independent of fc, and admits a probability density 
TTs{s) with support [0, L], and such that iisis) > c^ > for s G [0,i]. 

Theorem 4.2. For the FEXP prior described above, there exist C,C' > such that, for n large 
enough 



K 



pTT 



i{fJo)> 



Clogn 



„2/}/(2/3+l) 



< 



c 



where f = F(d, k, i 



EH 



eifJo) 



< 



C'jhgn) 

^2/3/(2/3+1) 



(4.2) 



(4.3) 



where log /(A) = E^ [log/(A)|X„]. 
A proof is given in Appendix [C] 



5 Proofs of Theorems 13.11 and 14.11 

5.1 Proof of Theorem I^TTl 

For the sake of conciseness, we introduce the following notations: for any pair (/, /o) of spectral 
densities, 

A{foJ) = T„(/)-ir„(/o), 

BifoJ) = T„(/o)i/2[T„(/)-i-T„(/o)-i]r„(/o)i/2. 



The proof borrows ideas from iGhosal et al.l (2000|). The main difficulty is to formulate con- 



straints on quantities such as hn{f, /o) or KLn{f, /o) in terms of distances between /, /o, inde- 
pendent on n, and uniformly over /. One has 

..,.|^, ^ /]1.4g(/)y>(X„;/)/v.(X„;/o)d^(/) a N^ 
^^' "^ /^(X„;/)MX„;/o)d7r(/) A/ 

Let 5 e (0, e) and P^ be a generic notation for probabilities associated to the distribution of X„, 
under the true spectral density /o ~ F^do^go). One has 

Po" {P^ [^J|X„] > e-"^} < Po" [A. < e-"^] + Po" [iV„ > e'^"^] (5.I) 

The following Lemma bounds the first term. 
Lemma 1. There exists C > such that 

Pa [Dn < e-"^] < Cn-^. (5.2) 

Proof. Lemma [4] implies that, when n is large enough, S„ D Bs/s, where 

Bn - {id,g) e [-l/2 + t,l/2-t] X gim,M,L,p) : KLMo,F{d,g)) < 6/4}. 

and Condition 2 implies that, for n large enough, Tr{Bn) > TriBs/s) > 2e~"''/^. Consider the 
indicator function 

a. = ll[-X*,{T„(/)-i -r„(/o)-i}X„ + logdetA(/o,/) > -nS] , 

with implicit arguments (/, X„), then, following IChosal et al.l (J2000l) . 

< Po" (i?'' {a„llgj/)} < 7r(^„)/2) 

< Po" [e^ {(1 - f7„)lle„(/)} > vr(i3„)/2) 

7r(o„) Jb,. 
by Markov inequality. Besides, 

E^o'll-^n} = Po"{X^T„(/)-i-T„(/o)-i}X„-logdetA(/o,/)>n5} 
= Py {Y*P(/o, /)Y - tr [P(/o, /)] > i?(/o, /)} 
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where Y ~ iV„(0„,I„), and, for f £ Bn 



thus 



D{fa, I) = nS + logdet A{fo, f) - tr [B{fo, /)] > nS/2 



E'o'[l-n,,] < P^{Y'B{foJ)Y-tT[B{fo,f)]>n6/2} 



< 
< 



16 
C 



Ey 



{Y'B{fo,f)Y-tr[B{foJ)]Y 



n 



3<54' 



which concludes the proof. 



A bound for the second term in (15.11) is obtained as follows 



D 






(5.3) 



using Condition 3, where 



P — ^0 






Assuming 26 < r, we consider the following likelihood ratio tests for each fi G Tin, and for 
some arbitrary values pi, 

^. = 11 {X^ [T-'ifo) - T-\f,)] X„ > np,} . 

Lemmas [71 [5] and El given in Appendix iBl prove that, for each of the three cases in Condition 
3 of Theorem 13. II and well-chosen values of pi, one has 



KM 



<g-nOie^ £;"[1 



< e 



-nCie 



(5.4) 



for all fi, for / close to fi (in the sense defined in cases a,b, and c in Condition 3), where Ci > 
is a constant that does not depend on fi, and E^ stands for the expectation with respect to 
the likelihood <y9(X„;/). Then one concludes easily as follows. Let 0^"^ = maxi(f>i, then, using 
Markov inequality, for n large enough. 



p < K 



2e 



2nS 



Ef 



^ p —nCie I rf 2nS—nCi6 
< g-nCie/2 



1 - 0(")1 dn{f) 



(5.5) 



provided S < Cie/4. Combining (|5.2p . (|5.3p and (|5.5p . there exists 6 > such that 
for n large enough, which implies that P'^[A^|X„] -^ a.s. 
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5.2 Proof of Theorem [HI] 



This proof uses the same notations as the previous Section, e.g. C, C denote generic constants, /, 
dn{f) are short-hands for / = F{d,g), dTr{d,g), respectively, A{f, /o) and B{f, /o) have the same 
definition, and so on. In the proof of Theorem 1 3. 1[ we showed that Eq [P'^{h{f, /o) > £|X„)] < 
Cn~^ for e small enough, n large enough. Thanks to the uniform convergence Lemmas [3] and 
m in Appendix 13 one sees that the same inequality holds if h is replaced by /i„. Therefore, 
to obtain inequality (j4.ll) , it is sufficient to bound the expectation of the sum of the following 
probabilities: 

for lo < I < Im where 

W,u = {(rf, <?) : Hf, /o) < £, ell < hMoJ) < el{l + 1)} , 
and Vn,i = y^n,i n Fn- Following the same lines as in Section [531 one has 



E- 



2-^ n 



.l=lo 



< Po"CD„<e-"""/2 



-e;; 



g^lip„>e- 



U=lo 



D^ 



V2) 



(5.6) 



The first term is bounded as in Lemma [l] see Section \5A 



Po'M^n < e"""'V2 < Po[Dn< 



-7lUn/2 



ABn) 



< 



2j^^ES[{l-nM))]d7T{f) 



where r2„ is the indicator function of 

{(X„,/);X^(r-i(/) -T-i(/o))X„ -logdet[A(/o,/)] < n^„} , 

and, for f £ Bm using Chernoff-type inequalities as in Lemma [TJ together with the fact that 
there exists sq > fixed such that for all s < sq 

I„(l + 2s) - 2sr„(/o)i/2r„(/)-iT„(/o)^/2 > I„/2, 

for / = F{d,g), d > do, g > 0, we have for all < s < sq 

E'S [I - ^„] 

< exp{ -.s™„-slog|r„(/o)r„(/)-i| 

-i log |i„(i + 2s) - 2sr„(/o)i/2r„(/)-ir„(/o)i/2| | 

< exp {-sr7,M„ + 2snKLn{fQ, f) + 4s^t7,6„(/o, /)} 
- '^^^i 2~^ '^J 



< e 



-C7lUn 
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where the second inequahty comes from a Taylor expansion in s of log |I„+2s(I„— r„(/o)^/'^r„(/)^^T„(/o)^/^)|, 
the third from the definition of S„ and the last from choosing s = min(so, 1/16). Note that 
So > m/(M7r) and that the constant C in the above inequality can be chosen as C = m/(32M7r). 
The second term of (j5.6p equals 



E" 



l,^ 



Nn 



J2 T^^ll (^n > e-"""/2) {^i + l- 4'i) 



.l=lo 



< J2 ^0 [4>i] + 2e""" J2 ^0 [Nn,ia " ^i)] 



(5.7) 



l=lo 



where (/)i — maxj.y. ,g-^^ , '/>i,i, '/>i,i is a test function defined as in Section [5. II 

0,M - ll{X;(7^-i(/o) -T-i(/,,0)X„ > tr [I„ -T„(/o)T- 1 (/,,,)] + r^/^„(/o, /m)/4} • 
Using inequality (jB.2p in Lemma [71 one obtains: 

logE^i^u] < -Cn/^„(/o,/.)min ("^^44^,1 



for some universal constant C, and n large enough. In addition, one has 



(5.8) 



bnjfoji) 
hn{fo,fi) 



< 



Tnihf'^TM.)-^'^ 



<; /^' 2max(rfo-(ii),0) 



where the first inequalit y comes from Lemma El see Appendix lA.li and the second inequality 
comes from Lemma 3 in iLieberman et al.l (|2009f) . Hence for all C > 0, if 2|(io ~ rfi| < C/\ogn, 
bn{fo,fi) < C"e'"/i„(/o,/i). Moreover for all S > 0, there exists Cs > such that if 2|do — 
di\ > C5(logn)~^ then hn{fo,fi) > n^^ ■ Indeed, equation (jA.3|) of Lemma [5] implies that if 
hnUoJi) > el, then 

hnifo, fi) > ^tr [TMo')Tn{f^ " h)Tn{!^^)Tn{h ' /o)] 

and Lemma O see Appendix IA.3[ implies that, for all a > 0, 



-tr [T„(/o-i)T„(/, - fo)TMr')Tn{f. - /o)] - (27r)3 [^ iA^^ dX 
n J-TT JijQ 



< 71-^+". 



Lemma jTI] see Appendix |Dl implies that there exists a > such that ii 2\do — di\ > 
Cs {log nr\ 



ifi - foY 

fifo 



■da;>Ce-°'°sn/c. > „-5 



as soon as Cs is large enough. Choosing S < p we finally obtain that 

hn{foJ^)>C'n-'. 

This and the definition of 'Hn,i implies that I > C'n-^e-"^, and therefore ln-'^^'^i<>-«-'ii)^^) > 
2Z"/C", for all a < 1 as soon as jdo ~ di\ is small enough. (J5.8I) becomes 

log£;^' [0,,/] < -de^ni-^i^'^f^'o-'^-") < -2^11°". 
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Condition 13] implies that 

i 

SO that J2i Eq [4>i] < 2exp{— ne^/p } for n large enough. 

For the second term of (|5.7p . since condition [3] on /, fij implies that 

/ 2(<i,-d) 2|1ok|A|| 
\ 32 logn 

when n is large enough, hence trA(/i_/ — /, /o) < nhn{fo, fi.i)/^ and we obtain the first part of 
equation IB. 31 



We also have 

32 



bnif, /o) < 6„(/m: /o) + ^"^^g'^°^ + 2^bMo,ki)hn(Ai,fo), 



hence logi?*? [f — 4>i^i] < —cnl"e\, using the same arguments as before, and 

Y^ E^ [{I - 4>i)N^j] = / I £ llw„,,(/)i?/(l - h) \ dnif) 

i=io y.i=io ) 

< P^fe^^n{h{fJo)<e}) 



+E/ ^v„,,(/)i?7(i-0Orf4/) 

l=lo 

< e""^" + Y e"^"^"'° < 2e" 



., — n£„ 



6 Discussion 

In this paper we have considered the theoretical properties of Bayesian non parametric estimates 
of the spectral density for Gaussian long memory processes. Some general conditions on the 
prior and on the true spectral density are provided to ensure consistency and to determine con- 
centration rates of the posterior distributions in terms of the pseudo- metric hnifo, /). To derive 
a posterior concentration rate in terms of a more common metric such as I2 , we have considered 
a specific family of priors based of the FEXP models and also used in the frequentist literature. 
Gaussian long memory processes lead to complex behaviours, which makes the derivation of con- 
centration rates a difficult task. This paper is thus a step in the direction of better understanding 
the asymptotic behaviour of the posterior distribution in such models and could be applied to 
various types of priors on the short memory part - other than the FEXP priors. 

The rates we have derived are optimal (up to a log n term) in Sobolev balls but not adaptive 
since the estimation procedure depends on the smoothness /3. Another constraint in the paper 
is that the prior needs to be restricted to Sobolev balls with fixed though large radius, forbid- 
ding the use of Gaussian distributions on the coefficients appearing in the FEXP representation. 
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However, it is to be noted that even in the parametric framework existing resuhs on the asymp- 
totic behaviour of Ukelihood approaches, whether maximum hkelihood estimators or Bayesian 
estimators are ah assuming that the parameter space is compact, for the same reason that we 
have had to constraint the prior on fixed Sobolev balls in the FEXP example. The reason is that 
the short memory part of the spectral density needs to be uniformly bounded. 

A related and fundamental problem is the practical implementation of the model described 
in the paper. iLiseo and Rousseaul (|2006l ) adopted a Population MC algorithm which easily deals 
with the trans-dimensional parameter space issue. We are currently working on alternative 
computational approaches. 
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A Technical Lemmas on convergence rates of products of 
Toeplitz matrices 

We first give a set of inequalities on norms of matrices that are useful throughout the proofs. We 
then give three technical lemmas on the uniform conv ergence of traces o f prod ucts of Toeplitz 
matrices, in the spirit of iLieberman et al.l ( 2003 ) and iLieberman et al.l ( 20091 ). but extending 
those previous results to functional classes instead of parametric classes. 



A.l Some matrix inequalities 

Let A and B he n dimensional matrices. We consider the following two norms: 

|^|2 = tr [A^*] , ||Af = sup x'^AA'x. 

We first recall that: 

|tr[^i3]| < |A||B|, \AB\<\\A\\\B\, \A\ < \\Al \\AB\\<\\A\\\\B\\. 
Using these inequalities we prove the following basic Lemma: 
Lemma 2. Let /i,/2 be two spectral densities, then 

Proof. One has 

2nbMiJ2) 

< l|T„(/2)-^/^T„(/i)i/2f |r„(/2)-i/2r„(/i - /2)T„(/2)-i/2 

= n\\T^{h)-^'^T,,Uif'YK{hj2). 



tr 



T„ 



D 
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A. 2 Uniform convergence: Lemmas [3] and [3] 



We state two technical lemmas, which are extensions of iLieberman et alJ (|2003l ) on uniform 
convergence of traces of Toeplitz matrices, and which are repeatedly used in the paper. 

Lemma 3. Let t > 0, Af, L > and p G (0, 1], let p be a positive integer, we have, as n 

/:.nLi/.(A)//(A)dA 



-oo; 



sup 

fi=F(di,g,)J[ = F{d2,g[) 

2p{di+d2)<l-t 

gi&g{-M,M,L,p) 

g'.eg{-M,M,L,p) 



1 

-tr 
n 



n^"(/or„(/:) 



.i=l 



(2^)1- 



2p 



0. 



This lemma is a direct adaptation from ILieberman et alJ (|2003[ ): the only non obvious part 
is the change from the condition of continuous differentiability in that paper to the Lipschitz 
condition of order p. This different assumption affects only equation (30) of ILieberman et al. 
(|2003l ). with rjn replaced by 77^, which does not change the convergence results. 



Lemma 4. Let t > 0, M,L,m > and pi,p2 G (0,1], let p be a positive integer, we have, as 

n -^ +00 .• 



sup 

fi=F(di,a,) fl=F(d2,g[) 

ip{di-d2)<P2 + l-t 

gi&Q{-M,M,L,pi) 

g'-eg{m,M,L,p2) 



1 

-tr 
n 



tlTnimMir' 



.4 = 1 






-[ f]^ 



dX 



0, 



Proof. This result is a direct consequence o f Lemma B l as in ILieberman et al.l ( 2003 ). The only 
difference is in the proof of Lemma 5.2. of iDahlhaud ( 1989f) . i.e. in the study of terms in the 
form 

|i„ - r„(/)i/2r„ ((47rV)~') r„(/)i/2|, 

with / = F{d2,g[) for any i < p. For simp li city's sake we write / — F{d,g) in the following 
calculations. Following Dahlhaus's iDahlhausI (|l989l ) proof, we obtain an upper bound of 



/(Al) 



/(A2) 



which is different from IPahlhaud (19891). If 5 G Q{m,M,L, P2), the Lipschitz condition in p2 
implies that 



/(Al 



/(A2 



<K[ |Ai-A2|''^ + 



|Ai 



X2\'-' 



|Ai 



11-5 



Calculations as in Lemma 5.2 of lDahlhausI ( 19891 ) imply that 



|/ - T„(/)1/2t„ ((47rV)-') T„(/)i/2|2 = oin'-P- logn') + 0{n'), V<5 > 0. 
From this we prove the Lemma following ILieberman et al.l ( 20091 ) Lemma 7, the bounds being 



uniform over the considered class of functions. 



D 



A. 3 Order of approximation: Lemma O 

In this section we recall a r esult given in lKruiier and RousseatJ ( 20101 ) which is a generalization 
of ILieberman and Phillipsl (|2004l ) concerning the convergence rate of 



tr 



Y[Tn{fj)Tnigj 



/n- (27r)" 



n/,(A)g,(A)dA 



16 



Lemma 5. Let 1/2 > a > 0, L > 0, Af > and < p < 1, then for all S > there exists C > 
such that for all n G N* 



sup 

p{di+d2)<a 
g,,g'^eg{-M,M,L,p) 



1 



-tr 



p 



-(2^) 



2p-l 






< C'„-P+'5+2pa+ ^ 






(A.l) 



where di,d2 > —1/2 and 0+ = max(a,0). 



A. 4 Some other approximations: Lemma [6] 

Lemma 6. Let fj, j G {1,2} be such that fj{X) ~ F{dj,gj), where dj £ (—1/2,1/2), < m < 
gj < M < +00 for some positive constant m, M and consider b a bounded function on [— 7r,7r]. 
Assume that \di — cf2| < S, with S G (0, 1/4), then, provided di > ^2, 



-tr [r„(/i)~^r„(/i6)T„(/2)-iT„(/i6)] < C(logn) [|6|2 + ,S|6| 

^2, 

tr [TMr')Tn{fl - f2)Tn{f2')Tn{fl ~ /2)] 



^ 1 



(A.2) 



and, without assuming di > d2, 

1 
n 



< C 



hn{flj2)+n'-'^Whn{flj2) 



(A.3) 



Proof. Throughout the proof C denotes a generic constant. We first prove (|A.2p . To do so, we 
first obtain an upper bound on the following quantity: 



7(6) = -tT[TMl')Tniflb)TM2')Tniflb)] 



(A.4) 



First note that b can be replaced by |6| so that we can assume that it is positive. Since the 
functions gi are bounded from below and above, we can prove (|A.2|) by replacing fi by |A|^^'^*. 
Thus, without loss of generality, we assume that fi = \X\~^'^\ Let A„(A) — J21=i^^Pi~'^^J) 
and Ln be the 27r-periodic function defined by L„(A) = n if |A| < 1/n and i„(A) — |A|^^ if 
l/n < \X\ < TT. Then |A„(A)| < CL„(A), 



A„(Ai - A2)A„(A2 - A3)dA2 = 27rA„(Ai - A3), 



(A.5) 



and we can express traces of products of Toeplitz matrices in the following way. Let the symbol 
dX denote the quantity (iAidA2(iA3(iA4; the conditions on the ^j's imply 






/2(A2)/l(A4) 

A„(Ai - A2)A„(A2 - A3)A„(A3 - A4)A„(A4 - Xi)dX 
(27r)2 



6(Ai)6(A3)|A3|-'*A„(Ai - A3)A„(A3 - Ai)dAidA3 



-7r,7r]4 



6(Ai)fe(A3)|A3r^'> -i -i 



" A3 
A2 


-2d2 


Ai 

A4 


-2di 

-1 



d\. 



(A.6) 
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as di ~ d2 < 5. We decompose the following factor in the integrand: 



-2d2 



-2di 



-2d2 



-2d2 



- 1 



-2di 



(A.7) 



-2di 



-1+1 



and treat each corresponding integral separately. Starting with the first term, replacing A„ by 
Ln, we obtain: 



6(Ai)6(A3)|A3r'*A„(Ai - A3)A„(A3 - A^dAidAs 
b{\,)b{X3)\X3\-^'Ll{Xi - A3)dAidA3 
< CI b{Xi)b{X3)\X3\-^'L„{Xi~X3)dXid\3 



1 
< - 

n ./[-7r.7r]2 



< C^ I 62(Ai)L„(Ai-A3)dAidA3 

'{fc(Ai)>b(A3)|A3|-2n 



'{fc(Ai)<b(A3)|A3|-2* 
< C\ fb^(Xi)Lr,{X,~X3)dX,dX3 



62(A3)|A3r**L„(Ai-A3)dAidA3) 
} -' 



< 



+ J b^Xs) \\X3\-^' - 1| L„(Ai - A3)dAidA3} 
C{\ogn){\b\l+6\b\l}, 



using calculations similar to IPahlhaua ( 19891 . Lemma 5.2). 

For the integral corresponding to the second term in (jA.61) . we note first that for < a < 
1 -di < 1 -^2, 



-2di 



KC 



lAi - A. 



|l-a 



lAiP 



and the same inequality holds if Ai, A4 and di are re placed, respectiv ely, by A3, A2 and ^2- Using 
the same calculations as the proof of Lemma 5.2 in iDahlhausI ( 19891 ). one has 



/ 6(Ai)fo(A3)|A3r 



2(5 



-2d2 



1 



-2di 



1 X 



Ln{Xi — A2)L„(A2 — A3)L„(A3 — A4)i„(A4 — Xi)dX 

|2 f Ln{Xi — X2)Ln{X2 — A3)°L„(A3 — A4)L„(A4 — Ai)° 



< C\b\i 



|Ai|i-"IA3|i--+2^ 



d\ 



-7r,7rJ' 
|2 „2a/i„„^\2 



< C|6|l,n^°(logn)^ 



provided a > 26. Taking a = 3i5 < 1/2 and doing the same calculations for the integrals 
corresponding to the two intermediate terms in (jA.6|) . one eventually obtains, when n is large 
enough 

jib)<Cilogn){\b\l + S\b\l}. (A.8) 
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We now prove that, for large n and Va > 
1 



Let 



tr [T„(/i)-ir„(/i6)T„(/2)-ir„(/i6)] <C{^{h)+n--^] 



5n - iT[Tn{hh)T-\h)T.^{hh)T-\h)\ 

= tr [r„(/i&)r„(/2-V47r2)r„(/i&)r„(/rV4^')] 



-tr 
-tr 



TMih)T-\h)Tn{hh)T-^'\h)RiT-''^{hi) 



where R^ = I„ - r„(/,)i/22;^(j-i/47r2)2;^(j^)i/2^ i = 1,2. We bound the first term with (jXS 



-tr[r„(/i6)r„(/2-i)T„(/i6)T„(/r^)] < C{\ognf{K\l + 5\bU] 



Moreover 



tr 



T^Uib)T-\h)Tn{hb)T-^'\h)RiT-^'\h) 
< |i?i||r-i/2(gi)T„(/i6)r-i(/2)T„(/i6)T-V2(/,)| 



Lemmas 5.2 and 5.3 in iDahlhausI (119891 ) lead to, Va > 
tr 






Similarly, 



tr[r„(/ife)r„(/2)-i/2i?2T„(/2)-i/2T„(/i6)T„(/fV(4^')) 
< \R2\5'J'\\TM2)-'^'TMib)'^Y 

for all a > 0. Finally we obtain, when n is large enough 

Sn < Cn^'\b\oo6l/^ + C{\ognf {\br,\l + 6\b\^} , 



which ends the proof of (|A.2p . 

Wenowprove (TOll . Since fj > mhj := mlXl-^"^' , T-\fj) < T~\hj), i.e. T-^{hj)-T-\fj) 
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is positive semidefinite, and 

K{flj2) 



(A.9) 



= ^tr [r„(/i - h)T-\h)Tn{h - h)T-\h)\ 

> ^tr [TMi - f2)T-\h2)TMi - f2)T-\h^)] 
In 



~ 2n 



T„(/i - h)Tn\h2)Tn{h - f2)T-'/^ihi)RiT-'^'ih,) 



1 

2n 
1 



r„(/l - f2)T-\h2)TMl - f2)Tn 






n(87r2) 
1 



tr 



2n 



-tr 



1 



n(327r 



T„(/i - /2)T-l/2(/l2)i?2T-l/2(/^2)T„(/l - /2)r„ (/ir^) 
j-tr [T„(/i - f2)Tn{h^')TMi - /2)T„ (/ir')] 



(A.IO) 



.1/2. 



-l/M^2AArrl/2 



where Rj — !„ - Tj {hj)Tn{hJ /{Air ))Tn (hj). We first bound the second term of the r.h.s. 
of (jA.9[) . Let (5 > and e < Eq such that \d — d p] < 6 (Corollary [J implies that there exists such 
a value Eq). Then using Lemmas 5.2 and 5.3 of lPahlhausI (|l989r) 



tr [r„(/i - f2)T-\h2)TMi - f2)T-'/'{hi)RiT-'/\hi) 

< 2|i?i||r-i/2(/.i)T„(/i-/2)r-i/^(/.2)ir„(l/i-/2|)i/2T-V2(/,,)i| 

X ||T„(|A-/2|)1/2t,7V2(;,^)|| 
< Cn^'\T.-^/'{h,)TMi-f2)T-'/'(h2)\. 
Since hi < Cfi, 

|T-V2(/,,)r„(/i-/2)r-V2(/,)|2 = tr [T-\h,)TMi~f2)T-Hf2)TMi-f2)] 

< Ctr [T-\f,)TMi-f2)T-\f2)Tn{h~f2)] 
= Cnhn{fi,f2), 

and 
1 



tr 



T„(/i-/2)T-i(/2)T„(/i-/2)T-i/2(.gi)i?iT-i/2(5i)J < Cn^'-'^Whn{fij2). 
We now bound the first term of the r.h.s. of (|A.9p . 
1 



-tr 
n 



Tn{h~l2)T-''\h2)R2T-''\h2)TMl~f2)Tn{K^) 

< i|i?2||T-i/2(/,2)T„(/i-/2)r„(/.i)-i/2|||r„(/,i)^/'T„(;ir')rn(|/i-/2|)T-i/2(/j2) 



< 



< 



^^^^^^^^^Sil||T„(/^i)^/^r„(/ir')T„(i/i-/2i)T-i/2(/2 



x||T„(fai)- vX(IA- /2|)^/^||||r„(|/i-/2|)^/^r-^/^(/2 

< Cn''-'/^^hMiJ2), 
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Where the latter inequahty comes from Lemma 5.3 of lDahlhaua ( 19891 ) and from the fact that 



||r„(/ii)i/2r„(/i-i)i/2||2 ^ An^\\%,{h,f'^T,,{h^^/{An^))T^{h,Y'^\\ < \R,\ + 1 
Therefore, 



-tr [T„(/i-/2)r„(/i2-i)T„(/i-/2)T„(/iri)] < C /j„(/i,/2)+n-i/2+3V/i«(/i,/2) 
n L 

and, usmg the fact that C gj > fj, for j = 1,2 this proves (|A.9p . 



B Construction of tests: Lemmas [7], [8] and [9] 

Lemma 7. If8\do — di\ < p+1 — t (case a of Condition 1), the inequalities in (j5.4p are verified 
provided pi = tr [l„ - Tn{fo)T~^{fi)] /n + h„{fo, fi), f < fi and 

1 rf^^'^^^-f'^^^dX<hifo,n)/4. (B.l) 



2^ Jo /o(A) 

Proof. For all s G (0, 1/4), using Markov inequality, 

i?o"N < exp{-snpj£;o"[exp{-sX^{r-i(/,)-T-i(/o)}X„}] 
= exp I -snpi - - log det [I„ + 2sB(/o, fi)] 

< exp {-snp, - str [B{fo, /,)] + s^tr [((I„ + 2srB{fo, /,))-'B(/o, f^)f] } 

< exp {-snp, - str [B(/o, /,)] + ^sHv [B(/o, /.)'] } , 

where t G (0, 1), using a Taylor expansion of the log-determinant around s = 0, and the following 
inequality: 

I„ + 2stB(/o, /.) = (1 - 2st)I„ + 2sTT„{foy^^T„ify^TMo) > il„, 

since st < 1/4. Substituting pi with its expression, the polynomial above is minimal for 
Smin = hn{fo,fi)/8bn{fo,fi). Accordiug to Smin G (0,1/4) Or not, that is, whether hn{fo,ft) < 
2bn{fo,fi) or not, one has: 

-\0gESm < ^Mi22JJL^{hMoJ^)<2bMo,f^)} 

n l6o„(/o,/.i) 

^ _MAi/i)„,iJM^,2J. (B.2) 

16 Lo„(/o,/.i) J 

Since 8|do - c?j| < p + 1 - i, the convergences 6„(/o, fi) -^ fo(/o, fi) and /i„(/o, /i) -> /i(/o, /i) are 
unifom on the support of the prior tt, see Lemma 2. One deduces that, for any a > and n large 
enough, 

- log^o [<Pd < -77 ™in -^ , , ■ ■- , 2/i(/o, /») - a 



n 



l^^^"n&(/o,/0 + . 
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Since fi G A"^, h{fo, ft) > e, and one may take a — e /2 to obtain 

n 32 [b(fo,fi)+e''/2 

Since Ido—c^il ^ (p + 1 ^ ^) /8 < 1/4, Lemma fT2l see Appendix [Pl implies that there exists 
Ci > such that 

K[0^]<exp(-nCi£) 

for e small enough. 

If / is in the support of n and satisfies / < fi, and 8(d,; — d) < p + 1 — t, using the same kind 
of calculations and the fact that 

I„ - 2</2(/) {r-i(/,) - T-\fo)} r„i/2(/) > I„ + 2si?(/, /o), 

as r„(/) < T„(/,), we obtain for s e (0, 1/4), 

^7 [1-0,] < expjnsp, -str[B(/,/o)]+4Ar[B(/,/o)2]} 

< exp {-ns/i„(/o, /,) + str [A(/, - /, /„)] + Ashv [B(/, fof] } 

< exp {-ns/i„(/o, /,)/2 + 4^^ [B(/, /o)^] } 

where the last inequality comes from (|B.1[) . which implies tr [A{fi — /, /o)] /n < /i„(/o, /i)/2 for 
n large enough, uniformly in /, using Lemma 2. Doing the same calculations as above, for n 
large enough 

To conclude, note that f < fi and (jB.ip implies that 

1 r f/' , „/ 



< 6(/„/o) + M/o,/.)/2 

< (C + l/2)/i(/o,/,) 

according to Lemma [T^ One concludes that there exists Ci > such that E^i [1 — (pi] < e""'^^^. 

D 

Lemma 8. // 8{di — do) > p + 1 — t (case b of Condition 3), the inequalities (|5.4p are verified 
provided pi = tr [l„ - r„(/o)T^^(/0] /n + 2KLn{fo;fi), for any f such that f < /, and 

Note that for e small enough, if b{fi, f) < b{fo, fi)\ loge]"^, (IB. 41) is satisfied. 
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Proof. The upper bound of i?o ['/'«] is computed similarly to (|B.2I) so that 

" 4 [ bn(fa,fi) 

According to LemmalTTIand since 8{di—dQ) > p+l—t, there exists C > 0, such that 6(/o, /i) > C 
Using the uniform convergence results of Appendix 1X1 this means that bn{fo,fi) > Cf^, for n 
large enough, independently of fi . Using Lemma [131 there exists a constant Ci < 1 such that 
KLn{fo, fi) > Cibnifo, fi)- Thus, there exists C2 > such that 

^logE^iq^,] <-nC2b{foJ.d, 

and, for e small enough, and some C3 > 0, 

K[0,]<exp{-nC3e}. 
As in the previous Lemma, let h E (0, 1): 



fogiJf [1 



< (1 - h)np,/2 



--logdet 



I. 



(1 - h)TM)'^' {T-\f^) - T^\fo)} TM)'^' 



< (1 - h)npj2 ~ - logdet [I„ + (1 - h)B{f, /o)] 
= (l-;i)np,/2-logdet[^(/,/o)]/2 



--logdet 



I„(l -h) + hT-^'\f)T,,{fo)T-^'\f) 



Substituting pi with its expression, i.e. npi — Iogdetyl(/, /o) = logdet A(/i, /) and using the 
same kind of expansions as in the previous lemma, one obtains 

- \ogEJ [1 -(/.,] < - logdet[A(/„ /)] + (V2)tr [TMo) {T-\f^) - T-\f)}] 
-hnLKMo; f^) + hHr [{l„ - T-i(/)T„(/o)}' 

< llogdet[A(/„/)] 
n 

-hnLKMo■J^) + hhr [{l„ - T-\f)TMa)y 

< -^logdet[A(/„/)] + 

. fKLnifoJ^ KLMoJj) 
-''^'^WBifoJ)yn 4 

Note that we use the fact f < fi in the second line. 

Since log det A{fi, f) = log det {l„ + r„(/i — f)Tn{f)^^ } , using a Taylor expansion of log det 
around I„, we obtain that for n large enough 



-logdetA(/„/)<— / ^^dX 
n 27r J_^ f 



where a can be chosen as small as necessary. In addition, we use Lemma [T^l and the uniform 
convergence results of Lemmas [31 JH to obtain that: 
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{nKL^{hJi)f ^ nm\b{foJ^? - af 



tr[i?(/o,/)2] - 16^8Af4(6(/o,/) + a) 



and, since d> do and (IB. 4 



< 2b{foJ^)(l + ^^ 

hence, under the constramt (jB.41) . there exists Ci > such that, for n large enough, e smaU 
enough, 

i??[l-0.] < exp{-nCi5(/o,/,)}<e^"^ 

D 

Lemma 9. // 8(do — di) > p + I ~ t (case c of Condition 3), the inequalities (j5.4p are verified 
provided pi = logdet[T„(/i)T„(/o)"^]/n if 

2^ J- f^^^^'^^-iM^^^^''^"^' b{fJ.,)<b{f,Jo) (B.5) 

Note that for e > small enough if /(/j - f)f^^d\ < b{fi, fo)\\oge\^^, (|B.5I) is satisfied. 
Proof. For < h < 1, following the same calculations as in the two previous lemmas, we obtain 

ilogi?o"M < -(l-/i)np,/2 + logdet[^(/o,/.)]/2 

To 



--logdet 



I„(l -h) + /iT-i/2(/o)T„(/,)T-i/2(/o) 
< -nhKL,,{f,, /o) + hhT[B{f,Jof] < -e. 



Moreover, for all / < fi, satisfying 8{di — d) < p + 1 — t, using the same calculations as in the 
proof of Lemma [71 we bound log E^i [1 — 0^] by the maximum of 

{nKL„{f,, /o) - tr[A(/, - /, /o)]/2}^ 

and 

ft 1 

--KLM^Jo) + gtr[A(/, - /,/o)], 

where a is any positive constant and n is large enough. Using Lemma [T51 one has 

2 
77 777 

nKLM^Jo)>^:^^^^bif,Jo) 
and the constraints (|B.5p we finally obtain that there exists constant ci , Ci > such that 
Ef [1 - 0,] < exp{~2n{KLn{f^Jo) - tr[^(/, - /, f)]/2n) + 4s^nbn{f, /o)} 

^ g-ncifc(/i,/o) ^ g-nCie 

for £ small enough. D 
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C Proof of Theorem 112] 

We re-use some of the notations of Section [O] in particular, C, C denote generic constants. 
The proof of the theorem is divided in two parts. First, we show that 



E- 



PM/:M/,/o)> ^'^^^ 



^2/3/(2/3 + 1) 



X, 



<-^. (C.l) 



Second, we show that, for / G Fn, and n large enough. 



K{f, /o) < Cn-^^ logn ^ h{f, /o) < C'n- wt \ogn. (C.2) 

Since i{f,fo) < h{f,fo), see the proof of Corollary [5] in Section [31 the right-hand side in- 
equality of (jC.2[) implies that 

E'o' {E^ [£ifJo)\X^]} < C- ^°^" 



„2/3/(2/3+l) 



-£E-{P^{hMJo)> J%'^^,^ \^n 



2/3 _„ 

< Cn 23+ilogn + Cn , 



for large n, where i < -f oo is an upper bound for i{f, /o) which is easily deduced from the fact 
that /, /o belongs to some Sobolev class of functions. This implies Theorem 14.21 
To prove (|C.1|) . we show that Conditions [T] and [5] of Theorem 14. II are fulfilled for 

w„ = n-2^/(2/3+i)(logn). 

In order to establish Condition [H we show that, for n large enough, Bn D Bn, the set 
containing all the / = F{d, k,9) such that k > A:„, for fc„ = kon^^^'^^'^^\ d — Un'nr'^ < do < d 
and, for j = 0, . . . , k, 

\0j-doj\<ij + ir^^unn-'', (C.3) 

where a > is some small constant. Then it is easy to see that 7r(S„) > 7r(;B„) > exp{— nu„/2}, 
provided fco is small enough, since 7rfe(fc > fc„) > exp{— Cfc„logfc„}, and (jC.3p for all j implies 
that 

3=0 j=Q 

k / k 

i=o \i=i 

< L 

for n large enough, since Lq = X); ^Oi(j + 1)^'^ < L, and X^f^i I^Ojl is bounded according to 



Let / — F{d,k,9), with {d,k,9) € Bn- To prove that {d,k,0) £ S„, it is sufficient to prove 
that /i„(/,/o) < M„y4, since /?„( /, /n) = K Ljfo;f) + KLjf;fo), a.nd K L^if -Jo) > C6„(/o,/), 
using the same calculation as in iDahlhausI (|l989l p. 1755) and the fact that d < do. 

Since /o G S{I3,L), and for the particular choice of fc„ above. 



-t-oo 



Y,el^<L{h + l)-'^ (C.4) 
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and 



Let 



+00 



1/2,^^ ,1/2 



(C.5) 



/on (A) 



6„(A) 



ll-e 



iA|-2d, 



" exp E ^oj cos(jA) 



.3=0 






and 5„ = 1 — /on//- Then f — fo = fob-n + fg-n, where 6„ and 5„ are bounded as fohows. From 



(jC.51) . one gets that, for n large enough, |6„|oo < Ckn , and 



2/3 ^n 

° logn 



according to (jC.4p . In addition since 1 — a; < — logo;, for a; > 0, 

5n(A) < (do - d) fog(l - cos A) + E l^oj - dj 

< Cu„n"''(|log|A|| + l). 
Moreover, since tr {(A + -B)^} < 2ti-A^ + 2ti-B^ for square matrices A and B, one has 



1 

n 



hnifoJ) < -tT[TMobn)T-\f)TMobn)T-\fo)] 

1 

n 



+ -tT[TM9n)T-\f)TM9n)T-\fo)] 
n 

< C\ogn{\bn\l+Unn-''\bn\i,} 



+Cu„n 



2„~l~2a 



tr 



(T„(/(|fog|A|| + l))r-i(/))^ 



< CUn 



(C.6) 



where c may be chosen as small as necessary, since feg is arbritrarily large. Note that the first 
two terms above come from (jA.2p in Lemma |51 and the third term comes from Lemma 01 

To establish Condition [2] is straightforward, since the prior has the same form as in Section 
13.31 and we can use the same reasoning as in the proof of Theorem 13.21 that is, take, for some 
well chosen S, 

Tn^[id,k,e)eSiP,L) : \d-do\ <<5, fc<fc„} 
where fc„ — kin^^^^^^^^ so that, using Lemma [TUl 

vr {T^ n {/, hif, fo) < e}) <7Tkik> fc„) < e-^^" '°sk„ 
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for n large enough. Choosing fci large enough leads to Condition 2. 

We now verify Condition |3] of Theorem 14.21 Let £^ > w„ and Iq < I < In, and consider 
/ = F{d,k,e), {d,k,0) e Vn,i, as defined in Theorem O and /,,; = (2e)'^"F(dj, /c,6'0, where 
dependencies on / in di and Oi are dropped for convenience. If for some positive c > to be 
chosen accordingly \9j — 9ij\ < cle\/{k + 1), for j = 0, . . . , fc, one obtains 

^^''^^^ - (2e)'^"exp<'> '(0,-%)cos(jA) 




5(A) 

< (2e2)='^" 

and fij/f > 1 so that the constraints of Condition [3] of Theorem 14.21 are verified by choosing c 
small enough. The cardinal of the smallest possible net under these constraints needed to cover 
Vn,i is bounded by 

- , / 1 \ fL'k„^^-+^ 

C , < h ' ' ' 



since for alH \9i\ < L. This implies that 

logC„j < Cnu„ 

and Condition [3] is verified with ef^ — £§«„. This achieves the proof of (|C.1I) . which provides a 
rate of convergence in terms of the distance hn{-, •)• 

Finally, we prove (jC.2p to obtain a rate of convergence in terms of the distance h{-,-). Consider 
/ such that 

hnih, f) = ^tT [T-\fo)TM - fo)T-\f)T„if - /o)] < el 
2n 

Equation (jA.3|) of Lemma [S] implies that 

i-tr[r„(/o-i)T„(/-/o)r„(/-i)r„(/-/o)] < Ce„[£„ + n-i/2+^] 

< C-^. (C.7) 

We now prove that 

tr [TMo')Tn{f - fo)Tn{r')Tnif - /o)] 

-tr[TMo\f-fo))TM-\f-m < ^IJS^. 

for some small a > 0. By symmetry we consider only the case d > do. Let /iq = (1 — cosA)*^", 
h — {1 — cos A)"*, then //i < C, /q/io < C and |/ — /o|ft. < C for some C > 0, and it is sufficient 
to study the difference below. Note that the calculations below follow the same lines and the 
same notations as the treatment of 7(6) in Lemma |6l see Appendix El 
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-tr[T,,{ho{f - fo))Tn{h{f - fo))] 
n 

--tr [Tn{ho)TM - fo)Tn{h)Tnif - /o)] 
n 

- -- / (/ - fo)i\2)hoiX2)if - /0)(A4)MA4) f ^J4i4 - 1 

xA„(Ai - A2)A„(A2 - A4)A„(A4 - Xi)dX 
'- f if- fo)i\2)ho{Xi)if /o)(A4)MA4) (^ - 1 



< 



xA„(Ai - A2)A„(A2 - A3)A„(A3 - A4)A„(A4 - Xi)dX 
^^^°^"^ ^ |A2r^('^-*')|A4ri+''L„(Ai-A2)i+"dA 

■,7r]2 

lAip'' 



C 

n 



__,. |A2P''|A3r'^ 



< 



xL„(Ai - A2)L„(A2 - A3)L„(A3 - A4)"i„(A4 - Xi)dX 
C(logn)2 



C(logn) 



|A2r2('^-'^°)|Airi+'^L„(A2-Ai)dA 
lAil^'^ 



< 



ni- y[-...]3 |A2P'^|A3|— ^"^^^ ^ ^^^"^"^^^ " ^^^^^ 
C(logn)2 



provided d — do < a/4, using standard calculations and inequality (IA.6I) . Combined with (IC.7[) . 
this result implies that 

itr [T,, {hoif ~ /o)) 7^„ [K! - /o))] < Cel. 
Finally, to obtain (|C.2p . we bound 

|tr [T„(/io(/ - fo))Tn{h{f - /o))] - tr [T,,{hoh{f - f^f)] \ 



C 



{hoif - fo)} (Xi) 



X [{Hf - fo)} (A2) - {h{f - fo)} (Ai)] A„(Ai - A2)A„(A2 - Ai)dA 



< C 



+c 



-c 



{Hf - fo)} (Ai)(/ - fo){X2)[HX2) - /i(Ai)]A„(Ai - A2)A„(A2 - Ai)dA 
{hhoif - fo)} (Ai) [/o(A2) - /o(Ai)] A„(Ai - A2)A„(A2 - Ai)dA 
{hhoif - fo)} (Ai) [/(A2) - /(Ai)] A„(Ai - A2)A„(A2 - Ai)dA 



The first term is of order ©(n^^logn), from the same calculations as above. We consider the 
last term, but the calculations for the second term follow exactly the same lines. Recall that 
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/ — /le™, where w(X) — X^^o ^J *"'^^(-?'^) ^^ ^^^ necessarily continuously difFerentiable, e.g. when 
/3 < 1. Thus 



/(A2) - /(Ai) = {h{\2)-^ - MAi)-^] e'"^^^) + h{\^)- 



gU)(A2) _ gW(Ai) 



The first term is dealt with using (IA.6I) . leading to a bound of order (logn)^n^". For the second 
term, and fc < fcr,, 



/io(/ - /o)(Ai)[.9(A2) - .g(Ai)]A„(Ai - A2)A„(A2 - \^)d\ 



< C / ho\f-fo\{Xi) 



Y^ 9j (cos(jA2) - cos(jAi)) 



j=o 



i„(Ai - A2)L„(A2 - Xi)dX 



< C(logn) I ^|0,|j j f {ho\f-fo\}{Xi)dXi 

< C(logn) I ^ \0,\j I (^£ {hhoif - fof} (A)dA 



1/2 



where the latter inequality holds because /_ {ho/h} (X)dX is bounded when \d — do\ is small 
enough. The same computations can be made on /o so that for all a > 4|(i — dol we finally obtain 
that 



|tr [T„(/io(/ - fo))Tn{h{f - /o))] " tr [T,,{hoh{f ~ fof)] \ 



< C{\ogn)n'^ + i\ogn)J2jm + \%\) ( / 

j=o yi-^,-^ 



1/2 



9og{f-fof{X)dX 



Splitting the indices of the sum above into into |j' : j\9j\ < j^^^^O^} and its complementary, for 
some r, we get that 

J=0 j=0 j=0 

provided we take r — 3/2 — j3. Using the same computation for /g, one obtains eventually that, 
provided ^ > 1/2, 

/ hohifo - ffdX < Cel 
which achieves the proof. 

D Technical lemmas 

The three following lemmas provide inequalities involving 

Hf, /o) = 7^ rif/h - ifdX, hif, /o) = ^ fu/h - if^dX, 



27r 



2n 



f 



for / = F{d,g), /o = F{do,go), d,do £ (0,1/2), g,go £ g{m,M), for < m < M. 
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Lemma 10. For any e > 0, 



1 /4M^ "^/^^ 



|d-do|>£^M/,/o)>- 

IT \ m 



Proof. Without loss of generality, take d > do, then, since {x — l)^/a; > x/2 for x > 4, 



> 

TT \ m 



D 



Lemma 11. There exists C > such that, for any £ > 0, 

\d-do\>e^b{f,fo)>C-^/^'. 
Proof. If d > do, then, since (x — 1)^ > x^/2 for x > 4, 



b(f, /o) > ^^ r 11 (A-2('i-do) > 4M/m| A-4('i-'io) ^^ 



4 /4M\~'/'" 



> 

TT \ m 



Otherwise, if d < do, one has {x — 1)^ > 1/4 for < x < 1/2, so 

b{f,fo) > ^y"ll{A2('^°-'^)<m/2M}dA 



1 /2My'/'" 
~ 8tt \ m 



D 



Lemma 12. For any t e (0, 1/4), there exists C > such that 

d-do<^-T^b{f,fo)<Ch{f,fo). 

Proof. If d < do, the bound is trivial, since ///o < M/rmT'^'^'^°^'^\ Assume d > do, and let 
A > 1/2 some arbitrary large constant. Since {x — 1)^ < x^ for x > 1/2, one has 

]\,f2 fir 

h{f,fo) < ^M/,/o) + ^^^y ll{/(A)//o(A)>yl}A-4('i-*)dA 

< Ah{f, /o) + 7 / 11 1 A-2('i-'io) > ^™/M I \'^^^-^«)d\ 

2'n:m^ Jo ^ J 

< AM/,/o) + ^ '-Y±^^ , (D.l) 
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provided A > M/m and C" — Af^/27rm^. In turn, since (x — 1)^ > x'^/2 for x > 4, and 
assuming A > iM^/rn^, then A^^f''"''") > Am/M implies that ///o > Am^/M^ > 4, and 
(///o - l)'/o// > //2/o > AmV2Af2. Therefore 

/^(/,/o) > ^fi{x-^^''-''"^>Am/M}{f/fo-lf^dX (D.2) 

> (Am/Af )^~^/^(^"'''') /47rA. (D.3) 

One concludes the proof by combining (jD.ip with (jD.Sp and taking A = 4A'P /rn^. 

D 

The lemma below makes the same assumptions with respect to / and /o, but it involves finite 
n distances. 



Lemma 13. One has: 



d>do^ KL„{fo; /) > j^^bnifo, /)■ 



Proof. iDahlhaud (|l989i p. 1755) proves that KLn{fo; f) > C-%n{fo, f) where C is the largest 



eigenvalue oi Tnifo)T-\f) . In our case, fo/f < Af7r2('^-''o)/m, hence C'^ = m^/M^Tr^'^''-'^°l 

D 

The last lemma in this section applies to the FEXP formulation of Section 13.31 

Lemma 14. Let 

/o(A) = (2-2cosA)-''°exp{u;o(A)}, /(A) = (2 - 2cos A)-''exp{7«(A)} , 

then, for e € (0, 1/4), 

\d~do\ <e,\w~wo\ < e ^ HfJo) < Is. 

Proof. Without loss of generality, take d - do > 0. Then /o// - 1 < 2^p/ - 1 < (1 + log 2)e, since 
e^ < 1 + 2.T for X G [0, 1]. Moreover, since 2(1 — cos A) > A^/3 for A G (0, tt), one has 



/o(A) Jo - 1-2£' 

and, to conclude, as again e^ < 1 + 2a; for a; G [0,1], and e^^^^^°^^\l - 2e)^^ - 1 < lOe, for 
e < 1/4, 

' '^(M+m-2]dx_ 



''^■« = sX m^m-'^''-^'^""'''- 
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